Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Rewrite Syntax #2654

Merged
merged 452 commits into from
Nov 8, 2021
Merged

Conversation

deathaxe
Copy link
Collaborator

@deathaxe deathaxe commented Dec 18, 2020

fixes #611
fixes #736
fixes #1610
fixes #1611
fixes #2919
closes #2991

At it current stage the syntax may be considered a rewrite, while it has actually been improved step by step from the existing implementation. All existing test cases have been maintained while being reorganized into logical sections/groups and being extended heavily.

On the way going it has turned into a proof of concept to illustrate the power of ST4's syntax engine and push it to the limits to reveal its current possibilities.

Highlights

Adds Java 16 langage features

  • sealed classes
  • record classes
  • switch expressions
  • compact constructors

Closely follows Java 13 Language Specification

The main goal is to match types and identifiers case insensitive whenever possible to fix as many ambiguities as possible which arise from current casing heuristics.

class a<b> implements intf<? extends test<b>> {
    string c() {
        e.f.a<b> c = (e.f.a<b>) new e.f.a<b>("foo");
        return c.say();
    }
}

grafik

That's achieved by using ST4's branch point feature to closely follow Java 13 Language Specification.

Overcome ST's next-line-blindness

All expressions can be wrapped wherever it makes sense without breaking syntax highlighting. No matter whether fully qualified identifiers, array modifiers, or function parameter groups all of them can be placed to onto following lines.

   class {
       void main() {
           h
           .
           e
           .
           l
           <
           ?
           extends
           @
           l
           .
           o
           (
           m
           =
           y
           )
           w
           >
           o
           []
           =
           new
           r
           (
           )
           .
           l
           .
           d
           (
           )
           ;
       }    
   }

grafik

Meta Scopes

With help of branching we now can scope all parts of members with desired meta.function scopes, even if those parts are located on different lines or consist of sophisticated expressions with arbritary nested contexts needed to match them.

class ClassName {
    
    @Annotation public int<T> method(int args) {
//  ^^^^^^^^^^^^^^^^^^^ meta.function.modifier        
//                     ^^^^^^^ meta.function.return-type
//                            ^^^^^^ meta.function.identifier
//                                  ^^^^^^^^^^ meta.function.parameters        
    }

    @Annotation 
//  ^^^^^^^^^^^ meta.function.modifier        
    public 
//  ^^^^^^ meta.function.modifier        
    int<T> 
//  ^^^^^^ meta.function.return-type
    method
//  ^^^^^^ meta.function.identifier
    (int args) 
//  ^^^^^^^^^^ meta.function.parameters        
    {
    }
}

grafik

Java Server Pages

The JSP syntax has been revamped using inheritance to replace with_prototype approach to inject JSP tags into HTML, CSS and JS with far less memory footprint.

<!DOCTYPE html>
<html>
<head>
    <title></title>
    <style type="text/css">

        @media <% print(myMedia) %> {
        }

        tr.<% print(myClass); %> {
            color: <% print("<\%foo%\>"); %>;
        }
    </style>

    <script type="text/javascript">
        <% if (true) { %>
            if (true) { <% print ("hello%\>") %> }
        <% } else { %>
            console.write("yep");
        <% } %>
    </script>
</head>
<body>
    <%-- This is a comment --%>
    <%@ include file="foo.bar" %>    
    <font color="<%=color.blue()%>"><%= map.size() %></font>
</body>
</html>  

grafik

Note:

It is not possible to maintain structure of two heavily context sensitive syntaxes such as Java and JavaScript when they are merged (the one embedded into the other). The decision was made to try to keep JavaScript intact and create Java in a way it may fail gracefully with loosing some meta scopes while trying to keep overall syntax highlighting intact. This may not work perfectly for all kinds of theoretically sophisticated edge case scenarios.

Miscellaneous

  • optimized to maintain syntax highlighting in incomplete statements
  • rewritten JavaDoc syntax
  • ...

Benchmarks

It's hard to make general statements about performance when branching is heavily used. Results heavily rely on content being used for benchmarks. While micro benchmarks of certain statements show significant slow downs overall impact in real world code seems to be less significant.

What does it mean in numbers?

Comparing the original syntax tests (current master) the new syntax seems 50% faster.

The largest source file of LemMinX is TreeLineTracker.java:

Description before now diff
Original file size (43kB / 1450 lines) 10.7ms 12.5ms +17%
Content duplicated (361kB / 10.1k lines) 66.6ms 82.4ms +24%

TODO

The syntax is at a point when it needs to be used in real world projects in every day work to find remaining issues.

We express non-popping contexts by plural
The pattern defined by this context is used only once.
Adds support for case-insensitive object types in local variable
declaration statements.
The current declarations context describes class member declarations.
Thus move it to class-body as it is used once only there.
Creates dedicated sections for global and member declaration contexts
and moves all related member contexts.
Try to find a consistent and common strategy to handle branches with
multi-push / multi-set statements.
We express non-popping contexts by plural.
Only object-types are supported, no primitives, no arrays.
Current solution was too inflexible.
Clarify the meaning of lookaheads.
1. Express optional contexts by `maybe-`
2. Singular context names express popping
3. Plural context names express non-popping
Removes `generic-` from context names as it not used by specs.
Don't break syntax if type identifier is missing
- verify fixes for sublimehq#1611
- make sure no details can break
Plural expresses non-popping.
- tweaks qualified identifier patterns
- tweaks parameter patterns
Don't include trailing whitespace into annotation identifiers.
Perform `declarations` branching only for tokens which look like identifiers as declarations always begin with such.

Catch assignments to improve performance in some situations.
@deathaxe deathaxe deleted the pr/java/rewrite-syntax branch September 25, 2021 13:31
@deathaxe deathaxe restored the pr/java/rewrite-syntax branch September 25, 2021 13:32
@deathaxe deathaxe reopened this Sep 25, 2021
michaelblyons
michaelblyons previously approved these changes Sep 25, 2021
@deathaxe
Copy link
Collaborator Author

Sorry, deleted the wrong branch by accident.

@michaelblyons
Copy link
Collaborator

@djspiewak and @ismell: Any interest in giving this PR a try?

@jrappen
Copy link
Contributor

jrappen commented Sep 29, 2021

djspiewak and ismell: Any interest in giving this PR a try?

also @lastsecondsave and @StrangeNoises as former contributors to the package

@StrangeNoises
Copy link
Contributor

I've been using it on my daily driver for java 17 dev for weeks. If I've been quiet it's because it's been working for me without issues. 😀 I don't think I have a role in accepting it (that would be a shock!) but I do look forward to an app update that includes it, hopefully sometime soon, so that's my +1 if it's worth anything.

@ismell
Copy link

ismell commented Sep 29, 2021

@djspiewak and @ismell: Any interest in giving this PR a try?

I'm no longer coding in Java. But from a quick glance it looks great!

@lastsecondsave
Copy link
Contributor

I also don't work with Java at the moment. I've checked a bunch of classes from JDK and this syntax is pretty much perfect. Although native keyword is considered invalid for some reason:
Screenshot from 2021-10-05 20-05-00

Both `native` and `strictfp` are valid method modifiers.
@deathaxe
Copy link
Collaborator Author

deathaxe commented Oct 6, 2021

Just and oversight.

@deathaxe
Copy link
Collaborator Author

Resolved a conflict with #2760.

Copy link
Collaborator

@FichteFoll FichteFoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is more than ready for a dev cycle. Any further edge cases can be ironed out later.

@BenjaminSchaaf BenjaminSchaaf merged commit 6b0eae5 into sublimehq:master Nov 8, 2021
@michaelblyons
Copy link
Collaborator

Hooray 🎉

@deathaxe deathaxe deleted the pr/java/rewrite-syntax branch November 8, 2021 19:39
@jwortmann
Copy link
Contributor

I think I found an example where the current highlighting is not optimal:

If you write a single method call or static function call on a line, and the next line is a variable declaration or assignment, then the method/function call will be scoped storage.type.class while typing. For the case of a variable declaration on the next line (excluding primitive types), the class name for that variable will even switch its color while you type.

Here is an example (I disabled the autocompletion for less distraction):

java

import java.util.ArrayList;

public class MyClass {

    public void myMethod() {
        ArrayList<String> myList = new ArrayList<String>();

        myList.add("foo");
        myList.add("bar");
        myList.add           // <- currently typing

        String str = "test"; // "myList" and "add" are storage.type.class in the line above, "String" is not
    }
}

There is a pretty strong convention in Java, that class names should start with an uppercase letter, and variables should start with a lowercase letter. Maybe the shown highlighting is what the compiler would see at the particular point in time, but it is not how I as a user would expect syntax highlighting to behave. So I would say the current highlighting for this example is a bad heuristic about what the user wants to write. Because I think it is extremely unlikely that the user really intents to write a variable declaration in the form of using both a lowercase class name and a linebreak between the class name and variable identifier, at the same time.

Could this perhaps be improved?

@jrappen
Copy link
Contributor

jrappen commented Feb 6, 2022

@jwortmann Not sure why you don't open a new issue. This PR was already merged.

mitranim pushed a commit to mitranim/Packages that referenced this pull request Mar 25, 2022
…blimehq#3018)

* [JavaScript] Refactor shebang highlighting

This commit refactors the way how shebang context is implemented after
the scheme of PHP/Java/ShellScript.

It makes sure to only highlight a comment as shebang in the very first
line. The only downside is it not being testable anymore.

* [TypeScript] Add xml highlighting in triple-slash comments

Fixes sublimehq#2880

This commit adds xml-tag syntax highlighting in triple-slash line
comments at the beginning of a document.

XML syntax is embedded in order to ...
1. enforce escaping on EOL
2. prevent any prototype from being injected into XML

* [TypeScript] Fix ligatures in triple-slash comments

This commit prepends a pattern to `comments` to consume tiple-slash
line comments in the middle of a typescript document in order to enable
ST to properly display triple-slash ligatures in the same way as they
are now displayed in directive comments at the beginning of a document.

* [JavaScript] Refactor comments

This commit...

1. moves triple-slash comment contexts to JavaScript in order to
   highlight comments equally in all JS/TS/JSX/TSX derivatives.
2. splits block and line comment contexts
3. applies some changes to highlight arbitrary amount of slashes
   or asterisks as comment punctuation. This is already applied to
   some other syntax definitions and intents to
   a) provide a more consistent highlighting in case a color scheme
      defines dedicated rules for comment punctuation.
   b) improve ligatures detection/grouping
4. A pattern for empty block comments is added to avoid complicated and
   probably error prone negative lookaheads in case of `/**/` or `/***/`
   scenarios in which the opening and closing punctuation can't be
   determined properly.

Notes:
1. The overall strategy with regards to block comments is borrowed from
   PR sublimehq#2654.
2. Line comments could be simplified if `double-slash`, `triple-slash`
   sub scopes are considered unnecessary.

* [JavaScript] Restrict shebang highlighting to very first line

This commit adds some prototype exclusions to ensure shebang comments
to be highlighted at the very first line only.

* [JavaScript] Scope trailing slashes in line comments

This commit is borrowed from Java to scope trailing slashes in line
comments as punctuation in case someone wants to use them as separator
with title.

* [JavaScript] Rename javascript context to script
mitranim pushed a commit to mitranim/Packages that referenced this pull request Mar 25, 2022
* [Java] Add method invocation tests

* [Java] Rename instantiation context

We express non-popping contexts by plural

* [Java] Update accessor_dot variable

* [Java] Remove variable-namespace context

The pattern defined by this context is used only once.

* [Java] Nondeterministic local variable declaration

Adds support for case-insensitive object types in local variable
declaration statements.

* [Java] Reorganize declarations as class-members

The current declarations context describes class member declarations.
Thus move it to class-body as it is used once only there.

* [Java] Reorganize declarations in sections

Creates dedicated sections for global and member declaration contexts
and moves all related member contexts.

* [Java] Set away from branch contexts

Try to find a consistent and common strategy to handle branches with
multi-push / multi-set statements.

* [Java] Rename statements contexts

We express non-popping contexts by plural.

* [Java] Update comments

* [Java] Scope all commas separator.comma

for consistency

* [Java] Rename parens -> group

* [Java] Fix array-only or object types in generics

* [Java] Fix exception list

Only object-types are supported, no primitives, no arrays.

* [Java] Reorganize object-type

Current solution was too inflexible.

* [Java] Add cast expression contexts

implements:

https://docs.oracle.com/javase/specs/jls/se13/html/jls-15.html#jls-15.16

* [Java] Rename variable branch contexts

* [Java] Refactor reference-type context

* [Java] Add variables to improve readability

Clarify the meaning of lookaheads.

* [Java] Move blocks section next to groups

* [Java] Rename generic type contexts

1. Express optional contexts by `maybe-`
2. Singular context names express popping
3. Plural context names express non-popping

* [Java] Rename generic type contexts part 2

Removes `generic-` from context names as it not used by specs.

* [Java] Improve incomplete type-parameter handling

Don't break syntax if type identifier is missing

* [Java] Extend generic local declaration tests

- verify fixes for sublimehq#1611
- make sure no details can break

* [Java] Add more ambiguity tests

- verify fixes for sublimehq#1611

* [Java] annotation context to annotations

Plural expresses non-popping.

* [Java] Improve annotation performance

- tweaks qualified identifier patterns
- tweaks parameter patterns

* [Java] Tweak annotation boundaries

Don't include trailing whitespace into annotation identifiers.

* [Java] Reorganize expression contexts

* [Java] Declaration vs. expressions performance tweak

Perform `declarations` branching only for tokens which look like identifiers as declarations always begin with such.

Catch assignments to improve performance in some situations.

* [Java] Use declarations in for statements

* [Java] Use declarations in try statements

* [Java] Simplify variable declaration identifier

Don't highlight unexpected keywords, just continue.

* [Java] Tweak variable identifier performance

The trick is as follows:

1. Push into branch if the next token looks like an identifier but
   without testing the whole list of reserved words.
3. Just match the simple `{{id}}` as all reserved words are expected to
   be matched before variables context.

* [Java] Add some incomplete for statement tests

* [Java] Reorganize assignment expressions

Introduces common rules for all kinds of assignments.

* [Java] Remove declaration-assignment branch

1. Recent changes reduced performance impacts.
2. Keeping this branch required additional variable identifier rules.

* [Java] Simplify method default context

* [Java] Move stray-group from blocks to statements

* [Java] Merge declaration sections

* [Java] Move member-declarations

Between class/enum/interface and field/method seems more logical.

* [Java] Rename member-statements

* [Java] Merge class-body into class-body-block-common

* [Java] Tweak field declaration contexts

* [Java] Add some field declaration tests

- ensure no overlapping meta.field appear
- add invalid var type test

* [Java] Tweak method declaration contexts

* [Java] Rename class/enum/interface modifier contexts

Plural expresses non-popping.

* [Java] Tweak package declaration

* [Java] Rename expect-type-reference -> maybe-...

"expect" is too strong as context just pops if no reference is matched.

* [Java] Make more use of group-body

* [Java] Make more use of block-body

* [Java] Rework instantiation-type

- highlight illegal modifiers/keywords
- use general scope for unexpected keywords, types and modifiers
- update block contexts
- update tests

* [Java] Tweak for parameters

Avoid popping group off stack, if unmatched tokens exist.

* [Java] Tweak module declaration

* [Java] Add tests to verify lowercase constructor

- verify fix for sublimehq#1610

* [Java] Fix hexadecimal floats

- add float suffixes to hex numbers
- reorganize literal tests

* [Java] Reorganize/Cleanup "Incomplete" Tests

Moves all unrelated to "incomplete" tests to their correct section.

* [Java] Recover from incomplete expressions and unclosed groups

Highlighting semicolon illegal and pop causes nested expressions not to
be popped off stack correctly. As a result an expression may be endless.

Hence the illegal-semicolon and illegal-open-block are replaced by
lookaheads.

* [Java] Rename illegal-type-parameters

Use singular to express popping contexts.

* [Java] Tweak expression-terminator

* [Java] Tweak annotation contexts

- reuse `annotation-parameters` context in qualified and unqualified
  annotations
- replace `(?=\S)` patterns by empty patterns in identifier like
  contexts as word boundaries are already maintained by the pushing
  pattern.

* [Java] Tweak enum-constant contexts

* [Java] More permissive case enum constant matching

Keep enum-constants highlighted even if `:` is missing.

* [Java] Move method reference expressions tests

* [Java] Move while statements tests

* [Java] Move for statements tests

* [Java] Move instance creation expressions tests

* [Java] Move type cast expressions tests

* [Java] Move incomplete method declarations tests

* [Java] Tweak local variable declarations tests

* [Java] Add labeled statement tests

* [Java] Add assert statements tests

* [Java] Fix word separators

Java identifiers may contain `$` for legacy reasons. Hence that
character is not part of word bounderies and `\b` is insufficient to use.

* [Java] Tweak identifier's word separator

* [Java] Give variables branch up immediately

* [Java] Use entity.name.tag in JavaDoc

see: sublimehq#2569

* [Java] Rename wildcards scope

This commit scopes `*` as `variable.language.wildcard` because

1. it matches anonymous variables `_` in Python and Scala.
2. Makefile scopes asterisks `variable.language` already.
3. it is used in recently reworked CSS
4. by scoping it `keyword.operator` a wildcard in general might not
   easily be distinguished from arithmetic operators.

Note:

An alternative scope was `constant.other.placeholder`.

* [Java] Scope extends/implements as storage.modifier

That one fits better than keyword.declaration as those keywords denote
modification to an existing data structure.

* [Java] Scope invalid extends/implements unexpected-keyword

* [Java] Use keyword.declaration.function

* [Java] Exclude prototype from meta-only contexts

* [Java] Fix JavaDoc Base Syntax

* [Java] Simplify groups patterns

Uses `set` within `branching` to match `(` only once.

Needs ST 4092+

* [Java] Remove obsolete reserved-pop contexts

* [Java] Rename enum-constant to constant

The section defines a pattern to match qualified or unqualified
constants. The prefix `enum-` might cause confusion with those belonging
to the enum declarations section.

* [Java] Tweak case label statements

1. Scope label part as `meta.case.label`.
2. Common `case-label-end` context.
3. Add case labels to symbol list to support working with huge state
   machines.

* [Java] Fix tmPreferences indentation

* [Java] Tweak illegal highlighting in catch groups

* [Java] Tweak anonymous function tests

* [Java] Rename meta.method to meta.function

* [Java] Remove Symbol List - Method

1. Function names are added to symbol list by default.
2. Function/method names consist of unqualified identifiers only, thus
   don't need any symbol transformation.

* [Java] Tweak line comment punctuation

* [Java] Move test files into tests directory

* [Java] Move JavaDoc tests to the top of the file

I just like comments to be tested first as their contexts are at the
top of the syntax file as well.

* [Java] Reduce Java/JavaDoc coupling by using embed

Avoid injecting JavaDoc contexts directly into Java by replacing the
include statement by embed. This reduces Java cache file by the size of
JavaDoc as Java now uses a reference of JavaDoc instead of a copy.

* [Java] Rename documentation comment scopes

* [Java] Make use of HTML prototypes

* [Java] Simplify JavaDoc tag names

Don't need a sub scope for each tag.

* [Java] Remove meta.block-tag.javadoc

The meta scope covers everything beginning with the first @... keyword.
That's a bit useless then.

* [Java] Rework inline javadoc tags

This commit...

1. removes `punctuation.definition` scope from `@` in tag names as it is
   considered not needed during discussions.
2. scopes opening and closing inline-tag braces as
   `punctuation.definition.tag.[begin|end]`
3. scopes inline tags as `meta.tag.inline.javadoc` as counterpart to
   `meta.tag.html`

* [Java] Add various JavaDoc tests

* [Java] Re-add meta.tag.block.javadoc

The meta scope covers the block tag name only to follow the scheme of
HTML tags.

* [Java] Add javadoc-inline-tag-end context

* [Java] Add some notes about JavaDoc

* [Java] Fix syntax name

* [Java] Add java syntax highlighting to {@code ...}

* [Java] Remove obsolete syntax_test_java from root

Was readded by merging master because Sublime Merge can't handle merge
conflicts with moved/deleted files properly. Due to too many differences
it was not seen this file was added again.

* [Java] Add description to Java (for JavaDoc)

* [Java] Simplify JavaDoc asterisk

* [Java] Remove expressions from declaration branches

ST seems to gracefully continue with the next pattern of the context
which includes `declarations` in case no branch succeeds.

* [Java] Tweak branch lookaheads

* [Java] Improve array item access meta scopes

* [Java] Named string body contexts

* [Java] Enable angle bracket matching

* [Java] Named ternary expression contexts

* [Java] Tweak member declaration contexts

* [Java] Add local variable declaration section

... for variable only use cases such as for/try parameter lists.

* [Java] Rework lambda parameters accordingly

Follows local variable declarations.

* [Java] Express optional character of variadic contexts

* [Java] Rename support.class to storage.type.class

As `support` is primarily meant for builtin or stdlib data types, a
general class shouldn't use it. Now that `class` uses `keyword.declaration` is seems safe to use `storage.type.class` to
denote user defined class data types.

* [Java] Add named comment body contexts

* [Java] Add named receiver identifier end context

* [Java] Tweak array modifiers after identifiers

* [Java] Add named accessor contexts

* [Java] Tweak object types childs

... used in code blocks.

* [Java] Remove expression from member branches

Obsolete as ST now handles failing branches well.

* [Java] Add support for multi-line qualifiers in code blocks

* [Java] Improve illegal type parameters

* [Java] Reorganize primitive and var types

* [Java] Rename additional bounds operator

* [Java] Rename function reference context

* [Java] Rename _ variable

* [Java] Tweak expressions

* [Java] Use accessor variable

* [Java] Improve type comparison expressions

* [Java] Refactor char and string literal contexts

* [Java] Scope import as keyword.declaration.import

* [Java] Codestyle

* [Java] Fix array modifier in incomplete expressions

* [Java] Fix variable highlighting in embedded code

Consider an `embed`ed section which ends with a variable. We might want
to highlight it `variable.other` rather than `variable.namespace`.
That's what this commit is for.

Example:

<jsp:expression>variable</jsp:expression>

* [Java] Rework Server Pages (JSP)

* [Java] Tweak receiver end context

... to match annotation-parameter strategy.

* [Java] rename var scope to storage.type.variant

Make clear it is a variant data type but no keyword.declaration!

* [Java] Tweak variable declaration context names

* [Java] Add specs reference for language literals

* [Java] Tweak code style

* [Java] Rename contexts from ...-body-block to ...-block

* [Java] Remove block-body context

* [Java] Remove group-body context

* [Java] Add named array dimensions and initializers

* [Java] Rename context tails from -content to -body

* [Java] Tweak symbol list scopes

* [Java] Tweak illegal stray scope names

* [Java] Refine meta.declaration

Distinguish type, identifier, value parts the same way fields do that.

* [Java] Tweak context names

* [Java] Move variable declaration contexts

* [Java] Remove .readwrite from variable scopes

* [Java] Use constant.other in local variable declarations

* [Java] Tweak context names

* [Java] Fix JSP test cases

* [LaTeX] Fix Java test cases

* [Java] Enable completions in try with resource statements

* Revert "[Java] Add java syntax highlighting to {@code ...}"

This reverts commit e8fdd84.

A code tag may contain any kind of syntax. Thus highlighting the content
as Java only is wrong.

* [Java] Fix block comment line start asterisks

This commit fixes an issue with the first `*` in normal non-javadoc
block comments not to be highlighted.

* [Java] Rescope anonymous function declaration operator

This commit renames anonymous function declaration operator to `keyword.declaration.function.arrow` with regards to
sublimehq#2617 (comment)

Notes:

1. The function body is still scoped `meta.function.anonymous` because a
   lambda function is anonymous regardless the keyword or operator being
   used to declare it.

2. The sub-scope `anonymous` is chosen, because it may be used as
   common name for anonymous functions and variables such as `_` in
   python and others.

3. The main reason for this commit is to provide separate scopes for
   lambda function operators such as `->` and `=>` and declaration
   keywords like `lambda` (Python) or `fun` (Erlang).

   Some users might want to assign different colors.

* [Java] Improve indentation rules

Standalone modifiers such as `protected:` are not present in Java's
syntax specification documentation and have not been found in real
world's code bases. Hence those are removed from indentation rules.

Otherwise `switch ... case` statements are supported now.
Conditional and loop statements' patterns have been improved to fix some
issues with block comments, strings and incomplete statements (missing `;`).

Indentation tests are added to verify correctness of rules. Those tests
concentrate on control flow statements. They may need to be extended by
class/enum/interface declarations and/or annotations etc.

* [Java] Simplify completion rules

1. A conditional or loop statement may be a one-liner.

Examples:

  if (v) { fun(); } else { bar(); }

  do { fun(); } while (bar());

We don't want to prevent completions after `do` or `else` then, even
though it is not good coding style.

2. Prevent completions when defining interfaces.

Example:

  interface Name {
     ...
  }

Note:

It seems not possible to disable completions for class/enum/
interface names in case line already contains further expressions:

  class Name| extends Other {
  }

  With `|` denoting caret, completion is still triggered due to
  `extends...`.

* [Java] Fix HTML integration

This commit fixes the no longer existing tag-custom context in
HTML.sublime-syntax, which was removed by PR sublimehq#2723.

* [Java] Update first line match pattern

* [Java] Make sure not to fail on JDK14 switch expressions

* [Java] Make sure not to fail on JDK14 instanceof patterns

* [Java] Add support for JDK15's sealed classes/interfaces

* [Java] Add support for JDK16 record classes

Add support for

   record ClassIdent (Type var, Type var2, ...);

Note:

Java calls the group after class identifier `RecordHeader` and the
content `RecordComponentList`, while this commit just applies the
`meta.parameters` scope, which is used in several places already.

* [Java] Add support for compact constructors

Required for record classes.

* [Java] Merge class- and enum-modifiers

Two contexts, same content.

* [Java] Improve readability of keyword variable lists

* [Java] Avoid duplicate look ahead patterns

Should save some roundtrips.

* [Java] Exclude prototypes from immediately popping contexts

* [Java] Use simple id pattern where safe

Avoids duplicated negative lookaheads patterns in situations when the
token is known not to be a reserved keyword due to the previous pattern,
which was executed. Should safe some roundtrips.

* [Java] Rename sealed class/interface tests

... to remove duplicate name for class vs. interface tests.

* [Java] Add support for yield-statements

Introduced with switch expressions.

* [Java] Refine case label meta scopes in arrow notation

* [Java] Split string literal contexts

We may be able to rework Groovy Syntax by extending Java with limited
effort, so let's start some preparations.

* [Java] Add dedicated literals context

Helps inheriting syntax definitions to inject literals.

* [Java] Add number suffix variables

May help inheriting syntax definitions to extend suffixes easily.

* [Java] More robust decimal point

Ensure not to match with Groovy's `..` range operators or `...` variadics.

* [Java] Rename accessor_dot to single_dot

A more general name for more general use.

* [Java] Reorganizes textblock contexts

This commit...

1. renames `multiline-strings` to `textblock` as this is what Java
   documentation calls triple quoted strings (""" ... """).
2. reorganizes them in a dedicated context for easier and selective
   replacement by inheriting syntax definitions.

* [Java] Reorganize tmPreferences

This package renames some tmPreferences to make clear which syntax
definition they belong to.

* [Java] Add @interface and record to cancelCompletion

* [Java] Update completions

This commit...

1. removes "contents" keys as those are equal to "trigger".
2. removes `goto` and `const` as those are illegal keywords.
3. adds `record`, `sealed`, `permits`, `yield`, `var` keywords
4. adds `null`, `false`, `true` constants
5. assigns `bool`, `char`, ... the "type" kind.
6. assigns `protected`, `public`, ... the "type" (modifier) kind.
7. assigns `suber`, `this` the "variable" kind.
8. sorts each group alphabetically

* [Java] Tweak tests formatting

* [Java] Detailed builtin constants scopes

Follows JavaScript

* [Java] Avoid duplicate look ahead patterns part 2

Should save some roundtrips.

* [Java] Exclude prototypes from immediately popping contexts part 2

Prototypes are not required in immediatelly popping contexts with
patterns, if it is known for comments or annotations to have been
matched by previous contexts on stack already.

* [Java] Tweak receiver branches

First pop, then set new contexts to reduce the number of contexts on
stack by one.

* [Java] Ensure meta.path boundaries in export statements

Ensure not to modify patterns in a way which break it.

* [Java] Rename meta.if scope

This commit scopes if statements `meta.statement.conditional` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

meta
  declaration
    variable
  statement
    conditional
    loop
    other
  expression

* [Java] Rename meta.else scope

This commit scopes else statements `meta.statement.conditional` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.switch scope

This commit scopes switch statements `meta.statement.conditional` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.case scope

This commit scopes case statements `meta.statement.conditional` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.while scope

This commit scopes while statements `meta.statement.loop` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.for scope

This commit scopes for statements `meta.statement.loop` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.assertion scope

This commit scopes assert statements `meta.statement.conditional` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.[break|continue|...] scope

This commit scopes flow statements `meta.statement.flow` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.synchronized scope

This commit scopes synchronized statements `meta.statement.flow` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.catch scope

This commit scopes catch statements `meta.statement.exception` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.finally scope

This commit scopes finally statements `meta.statement.exception` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.try scope

This commit scopes try statements `meta.statement.exception` for
consistency with Python and to provide some kind of logical meta scoping
for common tokens.

* [Java] Rename meta.typecast scope

This commit scopes typecasts `meta.cast` for consistency with C#.

* [Java] Rename ternary operator scopes

* Revert "[Java] Rename ternary operator scopes"

This reverts commit b469b7c.

* [Java] Highlight global/local function definitions

Required to maintain highlighting when embedding Java into other syntaxes.

* [Java] Reduce boolean scope granularity

* [Java] Reorganize embedded contexts in JSP

- renames meta.interpolation scopes to meta.embedded as this is what all
  the other syntaxes such as ASP/PHP/Rails use.
- rename context names accordingly.

* [Java] Add jsp string interpolation in CSS/JS

* [Java] Rename all embedded Java scopes in JSP

Finalizing scopes with HTML feels odd when those contexts are included
in JS/CSS.

* [Java] Fix meta.annotation.parameters boundaries

This commit fixes overlapping 
meta.annotation.identifier meta.annotation.parameters

* [Java] Improve catch() expression

This commit ...

1. scopes variables in catch arguments `variabe.other`.
2. scopes array modifiers illegal.

addresses sublimehq#2654 (comment)

* [Java] Use punctuation.section.generic

Addresses sublimehq#2654 (comment)

* [Java] Fix meta.annotation boundaries

This commit makes sure to not scope trailing newline of annotations
`meta.annotation` if no parameter list follows on next line.

* [Java] No class entity in static imports

* [Java] Exclude prototypes

* [Java] Improve emacs mode tag

Anything is allowed in front of `-*- mode: java -*-`

* [Java] Update java properties to sublime-syntax v2

* [Java] Scope file/page attributes in JSP as url

* [Java] Fix indentation after block comments

* [Java] Add indentation tests with line comments

* [Java] Fix toggle html comments in embedded css/js tags in JSP

* [Java] Scope arrow in switch statements separator

This commit...

1. modifies scope of arrows after case labels in switch statements to
   the same as colons in switch statements.
2. Shorten meta.scopes of `default` branch a bit by removing `.case`,
   so we end up with
     - `meta.statement.conditional.case` vs. 
     - `meta.statement.conditional.default`

Satisfy: sublimehq#2654 (comment)

* [Java] Add shebang highlighting

Closes sublimehq#2991

* [Java] Add JSTL variable interpolation support

see: https://www.oracle.com/technical-resources/articles/javase/code-convention.html

* [Java] Add file extension for Java Server Page Fragments

* [Java] Highlight namespace part of html/xml-tags

JSTL provides functions by namespaced custom tags. Therefore this commit
adds scopes namespace part of custom/unknown tags to make them visually
consistent with normal JSP tags.

They are still scoped `meta.tag.other` as it is not intended to start
long lists of "known" JSTL tags.

* [Java] Rename symbol list for fields

Uses plural for all files.

* [Java] Exclude local imports from global symbol index

Satisfy: sublimehq#2654 (comment)

* [Java] Tweak indentation rules comment

... to keep inline with CSS.

* [Java] Fix highlighting issue with constant case label

Fixes sublimehq#2654 (comment)

* [Java] Tweak case label end bailout

Exclude `{{variables}}` from bailout. Not sure why, but `super` or other
reserved variables should have caused the context to be popped, which should have caused some trouble.

Nevertheless, make sure it doesn't in future releases of ST.

* [Java] Fix case constant labels with annotations

This commit ensures not to break highlighting of case statements if the
label is preceded by an annotation.

* [Java] Improve qualified constants parsing performance

This commit increases parsing performance of qualified constants in case
statements by 820%.

Benchmark:

1. Create a `syntax_test_perf_constant_fqn.java` file
2. paste 100k lines of `case mod.sub.MyClass.EnumConst:` into the file
3. Run `Syntax Test - Performance`

Parsing times:

Before: 175ms
After:   32ms

* [Java] Tweak qualified constants parsing contexts

Turn `qualified-constant-class-or-name` into an immediately popping
intermediate management context without own patterns after the scheme
of `qualified-constant` context.

This doesn't change external behavior.

* [Java] Remove obsolete prototype contexts

* [Java] Tweak block comment patterns

* [Java] Restrict shebang highlighting to very first line

This commit adds some prototype exclusions to ensure shebang comments
to be highlighted at the very first line only.

* [Java] Fix allowed method modifiers

Both `native` and `strictfp` are valid method modifiers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet